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Abstract 

This paper describes ControlShell, a next- generation 
CASE “framework” for real-time system software de- 
velopment . ControlShelTs well-defined structure, graph- 
ical tools , and data management provide a unique 
component-based approach to real-time software gener- 
ation and management. ControlShell is designed specif- 
ically to enable modular design and implementation of 
real-time software. By defining a set of interface specifi- 
cations for inter-module interaction, ControlShell pro- 
vides a basis for real-time code development and ex- 
change. 

ControlShell includes many system-building tools , in- 
cluding a graphical data flow editor, a component data 
requirement editor, and a state-machine editor. It also 
includes a distributed data flow package, an execution 
configuration manager, a matrix package , and an object 
database and dynamic binding facility. ControlShell is 
being used in several applications, including the control 
of free-flying robots, underwater autonomous vehicles, 
and cooperating- arm robotic systems. 

This paper presents an overview of the ControlShell 
architecture, and details the functions of several of the 
tools. 

1 Introduction 

Motivation System programs for real-time command 
and control are, for the most part, custom software. 
Emerging operating systems [1, 2, 3, 4, 5] provide 
some basic building blocks — scheduling, communica- 
tion, etc. — but do not encourage or enable any struc- 
ture on the application software. Information binding 
and flow control, event responses, sampled-data inter- 
faces, network connectivity, user interfaces, etc. are all 
left to the programmer. As a result, each real-time sys- 
tem rapidly becomes a custom software implementation. 
With so many unique interfaces, even simple modules 
cannot be shared or reused. 
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An effective real-time framework must create a pro- 
gramming environment that facilitates sharing and reuse 
of real-time program modules. At a minimum, this re- 
quires providing interface specifications and data trans- 
fer mechanisms. The framework must also provide ser- 
vices and tools to combine modules and build systems 
from reusable components. Finally, the framework must 
meet the many challenges unique to real-time comput- 
ing. For example: 

• Real-time code must be able to react to external 
temporal events. 

• The real-time execution environment is fundamen- 
tally multi-threaded and asynchronous. 

• Real-time systems are usually composed of several 
different layers of control, each with different char- 
acteristics. For instance, strategic-level command 
and low-level servo control must be blended into a 
smoothly-operating system. 

• Real-time systems must handle changing condi- 
tions, often requiring switching between drastically 
different modes of operation. 

• Real-time systems are often physically distributed. 
In the simplest case, an operator control station 
may be remotely situated. More complex systems 
are comprised of many interacting distributed real- 
time and non-real-time subsystems. 

All these challenges must be efficiently and smoothly 
handled by the architecture. 

1.1 ControlShell’s Solutions 

Component-Based Design ControlShell is specifi- 
cally designed to address these issues. ControlShell pro- 
vides interface definitions and mechanisms for building 
real-time code modules. ControlShell also provides basic 
data structure specifications, and mechanisms for bind- 
ing data with routines and specifying data-flow require- 
ments. These two critical features make simple generic 
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packages (known as components) possible. ControlShell 
systems are built from combinations of these compo- 
nents. 

An extensive library of pre-defined components is pro- 
vided with the system, ranging from simple filters and 
controllers to complex trajectory generators and mo- 
tion planning modules. New or custom components are 
easily added to the system via the graphical Compo- 
nent Editor (CE). The Component Editor allows simple 
specification of data interchange requirements. Code is 
automatically generated to permit instancing the new 
component into the system. 

Graphical CASE System-Building Tools Con- 
trolShell also provides a set of powerful development 
tools for building complex systems. Building a system is 
accomplished by connecting components within a graph- 
ical Data Flow Editor (DFE). The data flow editor re- 
solves the system data dependencies and orders the com- 
ponent modules for most efficient execution. Radical 
mode changes are supported via a “configuration man- 
ager” that permits quick reconfiguration of large num- 
bers of active component routines. 

Real-time systems also require higher-level control 
functions. ControlShell’s event-driven finite state ma- 
chine (FSM) capability provides easy strategic control. 
The state machine model features rule-based transition 
conditions, true callable sub-chain hierarchies, task syn- 
chronization and event management. A graphical FSM 
editor facilitates building state programs. 

Real-Time System Services To provide support 
for real-time distributed systems, ControlShell includes 
a network connectivity package known as the Net- 
work Data Delivery Service (NDDS). NDDS provides 
distributed data flow. It naturally supports multiple 
anonymous data consumers and producers, arbitrary 
data types, and on-line reconfiguration and error recov- 
ery. 

ControlShell also offers a database facility, direct sup- 
port for sampled-data systems, a full matrix package, 
and an interactive menu system. Figure 1 presents 
an overview of the ControlShell toolset and design ap- 
proach. 

2 Relation to Other Research 

There are two quite different issues in real-time software 
system design: 

• Hierarchy (what is communicated) 

• Superstructure architecture (how it is communi- 
cated) 

Several efforts are underway to define hierarchy speci- 
fications; NASREM is a notable example [6], Control- 
Shell makes no attempt to define hierarchical interfaces, 



Figure 1: ControlShell Design Process 

The ControlShell system designer uses the many powerful 
tools , system services , and prebuilt library components to 
construct a modular system. 

but rather strives to provide a sufficiently generic soft- 
ware platform to allow the exploration of these issues. 
As such, this work takes a first step — defining the ar- 
chitecture superstructure (control and data flow mech- 
anisms). 

Several distributed data-flow architectures have been 
developed, including CMU’s TCA/X [7, 8], Rice Uni- 
versity’s TelRIP [9], and Sparta’s ARTSE [10]. These 
provide various levels of network services, but do not 
address repetitive service issues or resolve multiple data- 
producer conflicts in a symmetric robust “stateless” 
architecture as does the ControlShell NDDS system 
(see [11] for details). Also, they are not integrated 
within a general programming system. 

Recently, more sophisticated programming environ- 
ments have begun to emerge. For example, ORCAD [12] 
and COTS [13] are specialized robotics programming 
environments. Two commercial products, System Build 
with AutoCode from Integrated Systems, Inc. [14], and 
SIMULINK with C-Code Generation from the Math- 
Works, Inc. [15] are sophisticated control development 
environments. They offer easy-to-use rapid control sys- 
tem prototyping. They are not, however, architec- 
tures well suited to developing complex multi-layer dis- 
tributed control hierarchies. 

Implementation Experience ControlShell evolved 
from many years of research with real-time control sys- 
tems. It was first developed for use with a multiple- 
arm cooperative robot project at Stanford University’s 
Aerospace Robotics Laboratory [16, 17, 18]. From this 
start, ControlShell spread to become the basis for more 
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than 20 research projects in advanced control systems at 
Stanford. Among these were projects to study the con- 
trol of flexible structures, adaptive control, control of 
mobile robots (including multiple coordinated robots), 
and high-bandwidth force control [19, 20, 21, 22, 23, 24]. 
More recently, a few industrial sites and two NASA cen- 
ters have begun experimenting with ControlShell ap- 
plications [25, 26]. ControlShell is now being jointly 
developed by Stanford University and Real-Time Inno- 
vations, Inc. It is supported under ARPA’s Domain- 
Specific Software Architectures (DSSA) program. 

This continuous migration from specific, working ap- 
plications to wider spectrums of use is the key to usable 
generality. These applications continue to drive Control- 
Shell’s growth. To our knowledge, ControlShell is the 
only integrated framework package combining transpar- 
ent networking, component-based system description, a 
state machine model, and a run-time executive. 

3 Run-Time Structure 

Some of the major system modules are shown in Fig- 
ure 2. As shown in the figure, ControlShell is an open 
system, with application-accessible interfaces at each 
level. The figure is organized (loosely) into data and 
execution hierarchies. 

At the lowest layer, ControlShell executes within the 
VxWorks real-time operating system environment. The 
simple base class known as CSModules is the building 
block for most executable constructs. Organizations of 
these modules, into lists, menus, and finite state ma- 
chines form the core executable constructs. Users build 
useful execution-level atomic objects called components 
by defining derived classes from CSModules and bind- 
ing them through the on-line data base to data matrices 
from the CSMat package. High-level graphical editors 
speed component definition, data flow specification and 
state machine programming. Network connectivity is 
provided by NDDS for all application modules. 

4 Data Flow Design 

Many real-time systems contain sampled-data subsys- 
tems. Here, we define a “sampled-data” system as any 
system with a clearly periodic nature. Common exam- 
ples (each of which have been implemented under Con- 
trolShell) are digital control systems, real-time video im- 
age processing systems, and data acquisition systems. 
Each of these is characterized by a regular clock source. 

Providing an environment where sampled-data pro- 
gram components can be interchanged is challenging. 
These programs have routines that must be executed 
during the sampling process, routines to initialize data 
structures (or hardware) when sampling begins, and per- 
haps to clean up when sampling ends. Further, many 



Figure 2: Run-Time Structure 

ControlShell’s open architecture provides many powerful ser- 
vices, while allowing application code free access to the un- 
derlying structures . 

routines are dependent on knowledge of the timing pa- 
rameters, etc. Although they may interact — say by pass- 
ing data — sampled-data program components are often 
relatively independent. Requiring the application code 
to call each module’s various routines directly destroys 
modularity. 

4.1 Components 

The component is the fundamental unit of reusable data- 
flow code in ControlShell. Components consist of one or 
more sample modules derived from CSModules. Sample 
modules have several pre-defined entry routines, includ- 
ing: 


Routine 

When executed 

execute 

Once each sample period 

stateUpdate 

After all executes are done 

enable 

When this module is made active 

disable 

When it is removed from the active 
list 

startup 

When sampling begins 

shutdown 

When sampling ends 

timingChanged 

When the sample rate changes 

reset 

When the user types “reset”, or calls 
CSSampleReset 

terminate 

When the module is unloaded 

Thus, a motor 

driver component might define 


startup routine to initialize the hardware, an execute 
routine to control the motor, and a shutdown routine 
to disable the motors if sampling is interrupted for any 
reason. In addition, if any of its parameters depend on 
the sampling rate, it may request notification via a tim- 
ingChanged method. By allowing components to attach 
easily to these critical times in the system, ControlShell 
defines an interface sufficient for installing (and there- 
fore sharing) generic sampled-data programs. 
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Building Components: The Component Editor 
An easy-to-use graphical tool called the component edi- 
tor (CE) assists the user in generating new components 
and specifying their data-flow interactions. The com- 
ponent editor defines all the input and output data re- 
quirements for the component, and creates a data type 
for the system to use when interacting with the com- 
ponent. The tool contains a code generator; it auto- 
matically generates a description of the component that 
the Data-Flow Editor can display (see below), and the 
code required to install instances of the component into 
ControlShelPs run-time environment. 

4.2 Execution Lists 

An execution list is simply a dynamically changeable, or- 
dered list of sample modules to be sequentially executed. 
The active set of modules on a list can be changed any- 
time. In fact, lists may drastically change their contents 
during system mode changes. 

Execution lists may be sorted to provide automated 
run-time execution scheduling to resolve data dependen- 
cies. More specifically, the modules are sorted so that 
data consumers are always preceded by the appropriate 
data producers (see Figure 3). The system uses the spec- 
ifications of the data flow requirements for each compo- 
nent to sort the dependencies and order the list. A side 
benefit of the sorting process is the error-checking that 
is performed to insure consistent data flow patterns. 
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Figure 3: Dependency-Sorted List 

Dependency-sorted execution lists provide automatic run- 
time sorting by data dependencies. 


Sample Habitats ControlShell provides a named 
sampled-data environment, known as a sample habitat. 
A sample habitat encapsulates all the information and 
defines all the interfaces required for sampled-data pro- 
grams to co-exist. It also contains routines to control 
the sampling process. For example, a module installed 


into a sample habitat can query its clock source and 
sample rate, start and stop the sampling process, etc. 

Each sample habitat contains an independent task 
that executes the sample code. The task is clocked by 
the periodic source (such as a timer interrupt). Special 
components are provided to interface between habitats, 
allowing multi-rate controller designs. 

4.3 Building Systems: The DFE Editor 

Building systems of components is made simple by the 
graphical Data-Flow Editor (DFE). The DFE reads de- 
scription files produced by the component editor, and 
then allows the user to connect components in a friendly 
graphical environment. It allows specification of all the 
data connections in the system, as well as reference 
inputs — gains, configuration constants and other param- 
eters to the individual components. An example session 
is depicted in Figure 4. 



Figure 4: Data-Flow Editor 

The data-flow editor builds collections of components into 
an executing system. 


5 Configuration Management 

Complex real-time systems often have to operate under 
many different conditions. The changing sets of condi- 
tions may require drastic changes in execution patterns. 
For example, a robotic system coming into contact with 
a hard surface may have to switch in a force control al- 
gorithm, along with its attendant sensor set, estimators, 
trajectory control routines, etc. 

ControlShell’s configuration manager directly sup- 
ports this type of radical behavior change; it allows en- 
tire groups of modules to be quickly exchanged. Thus, 
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different system personalities can be easily interchanged 
during execution. This is a great boon during develop- 
ment, when an application programmer may wish, for 
example, to quickly compare controllers (See Figure 5). 
It is also of great utility in producing a multi-mode sys- 
tem design. By activating these changes from the state- 
machine facility (see below), the system is able to handle 
easily external events that cause major changes in sys- 
tem behavior. 


Configuration Hierarchy The configuration man- 
ager essentially creates a four-level hierarchy of module 
groupings. Individual sample modules form the low- 
est level. These usually implement a single well-defined 
function. Sets of modules, called module groups , com- 
bine the simple functions implemented by single mod- 
ules into complete executable subsystems. 

Each module group is assigned to a category. One 
group in each installed category is said to be active , 
meaning its modules will be executed. Finally, a config- 
uration is simply a specification of which group is active 
in each category. 



Figure 5: Configuration Manager 

Configurations can be swapped in or out under program or 
menu control. This provides flexible run-time reconfigura- 
tion of the execution structure. 


Example As a simple example, consider a system 
with two controllers: a proportional-plus-derivative con- 
troller named “PD”, and an optimal controller known 
as “LQG”. Suppose the PD controller requires filtered 
inputs, and thus consists of two sample modules: an in- 
stance of the PDControl component and a filter compo- 
nent. These two components would comprise the “PD” 
module group. The “LQG” controller module group 
may also be made up of several components. Both of 
these groups would be assigned to the category “con- 
trollers” . 


The user (or application code) can then easily switch 
controllers by changing the active module group in the 
“controller” category. 

Now suppose further that the controllers require a 
more sophisticated sensor set. A category named “sen- 
sors” may also be defined, perhaps with module groups 
named “endpoint” and “joint” . The highest level of the 
hierarchy allows the user to select an active group from 
each category, and name these selections as a configu- 
ration, Thus, the “JointPD” configuration might con- 
sist of the “joint” sensors and the “PD” controller. The 
“endptLQG” configuration could be the “endpoint” sen- 
sors and the “LQG” controller. 


Category and Group Specification This subdivi- 
sion may seem complex in these simple cases. However, 
it is quite powerful in more realistic systems. It has 
been shown to be quite natural in applications rang- 
ing from a vision-guided dual-arm robotic system able 
to catch moving objects [16] to flexible-beam adaptive 
controllers [27]. 

Assigning modules to groups and groups to categories 
is made quite simple with the ControlShell graphical 
DFE editor’s “configuration definition” window, shown 
in Figure 6. New categories are added with the click 
of a button. To create a module group, the user sim- 
ply names a group, and then clicks on the modules in 
the data-flow diagram that should belong to that group. 
The blocks are color-coded to relate the selections back 
to the user. 



Figure 6: Configuration Definition 

Configurations are easily defined within the DFE graphical 
interface. 
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6 Finite State Machines 

A real-time system in the real world must operate in 
a complex, event-driven environment. With only a se- 
quential programming language, the burden of manag- 
ing and reacting to events is left to the programmer. 

The Finite State Machine (FSM) module is designed 
to provide a simple strategic-level programming struc- 
ture that also assists in managing events and concur- 
rency in the system. The FSM module combines a 
non-sequential programming environment with natural 
event-driven process management. With this structure, 
the programmer is actively encouraged to divide the 
problem into small, independently executing processes. 

To utilize the FSM module, the programmer first 
describes the task as a state transition graph. The 
graph can be directly described within ControlShell ’s 
graphical FSM editor (see Figure 7). Each transition — 
represented by an arrow in the graph — specifies a start- 
ing state, a boolean relation between stimuli that causes 
the transition, the CSModule to be executed when the 
transition occurs, and a series of “return code-next 
state” pairs that determine the program flow. 

The FSM model is quite general; it supports rule- 
based transition conditions (reducing the number of 
states in complex systems), true callable sub-chains of 
states (so libraries of state subroutines can be devel- 
oped), wild-card matching (so unexpected stimuli can 
be processed), global matching (allowing easy error pro- 
cessing), and conditional succession (so state programs 
may easily branch). Transitions are specified as boolean 
relations of three types of stimuli: transient, latched, 
and conditional. Transient stimuli have no value, and 
exist only instantaneously. Latched stimuli also have 
no value, but persist until some transition expression 
matches. Condition stimuli have string values; they per- 
sist indefinitely and thus represent memory in the sys- 
tem. Thus, the transition condition “Object = Visible 
AND Acquire” might cause a system to react to an ac- 
quisition command from a high-level controller. Provid- 
ing these three stimuli types allows combination of both 
“system status” and “event” types of asynchronous in- 
puts into easily-understood programs. 

The FSM module takes advantage of the atomic 
message-passing capability of modern real-time kernels 
to weave the incoming asynchronous events into a single 
event stream. Any process can call a simple routine to 
queue the event; the FSM code spawns a process to ex- 
ecute the resulting event stream. The result is an easy- 
to-use, yet powerful real-time programming paradigm. 

7 Data Control and Binding 

Most data in a ControlShell application is embodied 
in CSMats . A CSMat is a named matrix of floating- 
point values. Each row and column of the matrix op- 



Figure 7: Finite State Machine Editor 

State transition graphs allow easy visualization of multi-step 
operations ; this example is a (simplified) program to catch a 
moving object with a duai-arm manipulator. 

tionally contains a field name and a units specification. 
A complete real-time matrix mathematics utilities pack- 
age is included. Components may combine multiple CS- 
Mats into structures for efficient reference and parame- 
ter passing. 

The entire control hierarchies are created and bound 
to the correct data objects at run-time. The system 
is built from the graphically-generated description files 
produced by the DFE and FSM editors. This dynamic 
binding paradigm is very powerful — it combines the con- 
venience of automatic system building with the flexibil- 
ity of a dynamically changeable system. Thus, it pro- 
vides the features of a full code generation without the 
pre-compiled inflexibility. 

To support this dynamic binding, ControlShell incor- 
porates a “linking” database facility. All instances of 
each data object (such as CSMats) — and each control 
construct (such as execution lists) — are entered into the 
database upon creation. The database allows “refer- 
ence before creation” semantics for many object types; 
if a requested object is not in the database (i.e. it does 
not exist), an incomplete (e.g. zero-sized) object will 
be created by the database itself. This capability allows 
considerable flexibility at run-time; modules may, for in- 
stance, specify dependencies on data sets that do not yet 
exist, etc. Verification routines insure that the system 
is consistent before actual “live” execution begins. 

8 Network Connectivity 

ControlShell is integrated with a network connectiv- 
ity package called the Network Data Delivery Service 
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(NDDS) [11]. NDDS is a novel network-transparent 
data-sharing system. NDDS features the ability to han- 
dle multiple producers, consumer update guarantees, 
notifications or “query” updates, dynamic binding of 
producers and consumers, user-defined data types, and 
more. 

The NDDS system builds on the model of information 
producers (sources) and consumers (sinks). Producers 
register a set of data instances that they will produce 
and then “produce” the data at their own discretion. 
Consumers “subscribe” to updates of any data instances 
they require. Producers are unaware of prospective con- 
sumers; consumers are not concerned with who is pro- 
ducing the data they use. Thus, the network configura- 
tion can be easily changed as required. NDDS is a sym- 
metric system, with no “special” or “privileged” nodes 
or name servers. All nodes are functionally identical and 
maintain their own databases. The routing protocol is 
connectionless and “quasi-stateless 1 ”; all data producer 
and consumer information is dynamically maintained. 
Thus dropped packets, node failures, reconfigurations, 
over-rides, etc. are all handled naturally. 

This scheme is particularly effective for systems (such 
as distributed control systems) where information is of 
a repetitive nature. NDDS is an efficient, easy-to-use 
distributed data-sharing system. Figure 8 illustrates the 
use of NDDS within a cooperating-arms robot system 
(see [24]). 



Figure 8: Network Data Delivery Service 

NDDS provides a network “backplane”. Each module can 
easily share data with any other module. The individual 
connections are handled transparently by the system. 


1 The databases at each node cache some state for efficiency, 
but aU information decays over time. 


9 Conclusions 

This paper has presented a brief overview of the ca- 
pabilities of the ControlShell system. ControlShell is 
designed — first and foremost — to be an environment 
that enables the development of complex real-time sys- 
tems. Emphasis, therefore, has been placed on a clean 
and open system structure, powerful system-building 
tools, and inter-project code sharing and reuse. 
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